Vision-based object recognition device and method for controlling the same

ABSTRACT

Embodiments of the present disclosure relate to vision-based object recognition devices and methods for controlling the same. According to an embodiment of the present disclosure, an electronic device may comprise a camera, at least one motor, a communication interface, at least one processor, and a memory electrically connected with the processor, wherein the memory may store commands that, when executed by the processor, cause the processor to identify a motion of an external object using a first image obtained by controlling the camera, obtain first direction information based on the identified motion of the external object, drive the at least one motor so that the camera faces a direction determined according to the first direction information, and identify a second electronic device from a second image obtained by controlling the camera facing the determined direction.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.15/900,980 filed on Feb. 21, 2018 which claims the benefit under 35U.S.C. § 119(a) of a Korean patent application filed in the KoreanIntellectual Property Office on Feb. 24, 2017 and assigned Serial No.10-2017-0025086, the entire disclosure of which is incorporated hereinby reference.

TECHNICAL FIELD

Embodiments of the present disclosure generally relate to vision-basedobject recognition devices and methods for controlling the same.

BACKGROUND

The Internet is evolving from the human-centered connection network bywhich humans create and consume information to an Internet of Things(IoT) network by which information is communicated and processed betweenthings (i.e. devices) or other distributed components. For example, anInternet of Everything (IoE) may be a combination of “Big Data” dataprocessing technology and an IoT network. In an IoE, connected devicemay be connected to a cloud server that performs “Big Data” dataprocessing.

An IoT network may include various components/technologies, such assensing technology, wired/wireless communication and networkinfrastructure, service interface technology, security, etc. Recentongoing research have been directed to inter-object connectiontechnologies, such as sensor networks, Machine-to-Machine (M2M)technologies, or Machine-Type Communication (MTC).

Also, the IoT environment may offer intelligent Internet Technology (IT)services that collect and analyze data generated by devices connected inthe IoT network. IoT networks may have various applications, such asbeing smart homes, smart buildings, smart cities, smart cars orconnected cars, other smart appliances, smart power grids, being used inhealthcare as intelligent medical devices, etc.

For example, in a smart home context, an IoT network may be used toconnect and control various home electronic devices. The user maycontrol various objects connected together via the IoT network by usinga master electronic device (e.g. the user's smartphone) that includes aprocessor and a communication interface. For example, the electronicdevice may control the various home electronic devices based on voicecommands received from the user.

Meanwhile, the plurality of objects connected together via the IoTnetwork may include objects of same or similar type, name, or function.For example, a plurality of lights, sound devices, or displays may bepresent in a household. These lights, sound devices, or displays may beconnected together via the IoT network. In such case, when the userwishes to control a particular object, he or she may be required tomanually distinguish the object from other similar objects, which may becumbersome. For example, where the electronic device receives the voicecommand “turn on the lamp” from the user, the electronic device may havedifficulty determining which one of the multiple lights connected in theIoT network should be turned on. In this case, the electronic device mayturn on the wrong lamp.

The above information is presented as background information only toassist with understanding the present disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

According to an embodiment of the present disclosure, an electronicdevice may identify the user's motion through vision-based recognition.The electronic device may then identify a particular object that theuser desires to control on the identified user motion. The electronicdevice may control the identified object based on commands received fromthe user. The electronic device may move or turn to the identifiedobject. The user may identify whether commands for controlling theobject have been delivered to the electronic device based on themovement or turn of the electronic device.

According to an embodiment of the present disclosure, an electronicdevice may comprise a camera, at least one motor, a communicationinterface, at least one processor, and a memory electrically connectedwith the processor, wherein the memory may store commands that, whenexecuted by the processor, cause the processor to identify a motion ofan external object using a first image obtained by controlling thecamera, obtain first direction information based on the identifiedmotion of the external object, drive the at least one motor so that thecamera faces a direction determined according to the first directioninformation, and identify a second electronic device from a second imageobtained by controlling the camera facing the determined direction.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise identifying a motion of an externalobject using a first image obtained by controlling the camera, obtainingfirst direction information based on the identified motion of theexternal object, driving at least one motor of the electronic device sothat the camera faces a direction determined according to the firstdirection information, and identifying a second electronic device from asecond image obtained by controlling the camera facing the determineddirection.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses exemplary embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the present disclosure and many of theaspects thereof will be better understood after referring to thefollowing detailed description of the drawings, wherein:

FIG. 1A is a block diagram illustrating an electronic device and anetwork according to an embodiment of the present disclosure;

FIG. 1B is a concept view illustrating an electronic device according toan embodiment of the present disclosure;

FIG. 2A is a block diagram illustrating an electronic device accordingto an embodiment of the present disclosure;

FIG. 2B is a block diagram illustrating an electronic device accordingto an embodiment of the present disclosure;

FIG. 3 is a view illustrating a method for controlling an electronicdevice using another electronic device according to an embodiment of thepresent disclosure;

FIG. 4 is a flowchart illustrating a method for enrolling an object tobe controlled using an electronic device according to an embodiment ofthe present disclosure;

FIG. 5A is a flowchart illustrating a method for controlling at leastone object using an electronic device according to an embodiment of thepresent disclosure;

FIG. 5B is a flowchart illustrating a method for identifying a result ofcontrolling at least one object using an electronic device according toan embodiment of the present disclosure;

FIG. 6 is a flowchart illustrating a method for identifying an objectthat is repositioned using an electronic device according to anembodiment of the present disclosure;

FIG. 7A, FIG. 7B and FIG. 7C are views illustrating a method for drivingat least part of an electronic device according to an embodiment of thepresent disclosure;

FIG. 8 is a flowchart illustrating a method for controlling anelectronic device according to an embodiment of the present disclosure;and

FIG. 9 is a block diagram illustrating a program module according to anembodiment of the present disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

Hereinafter, embodiments of the present disclosure are described withreference to the accompanying drawings. However, it should beappreciated that the present disclosure is not limited to theembodiments and the terminology used herein, and all changes and/orequivalents or replacements thereto also are within the scope of thepresent disclosure. It is to be understood that the singular forms “a,”“an,” and “the” may also refer to the plural, unless the context clearlyindicates otherwise.

As used herein, the terms “A or B” or “at least one of A and/or B” mayinclude all possible combinations of A and B. The terms “first” and“second” may refer to various components regardless of importance and/ororder and are used to distinguish a component from another withoutlimiting the components. It will be understood that when an element(e.g., a first element) is referred to as being (operatively orcommunicatively) “coupled with/to,” or “connected with/to” anotherelement (e.g., a second element), it can be coupled or connected with/tothe other element directly or via a third element.

As used herein, the terms “configured to” may be interchangeably usedwith other terms, such as “suitable for,” “capable of” “modified to,”“made to,” “adapted to,” “able to,” or “designed to” in hardware orsoftware. A device “configured to” perform certain operations may meanthat the device can perform the operations by itself or together withanother device. For example, the term “processor configured (or set) toperform A, B, and C” may mean a general-purpose processor (e.g., a CPUor application processor) that may perform the operations by executingone or more software programs stored in a memory device or a dedicatedprocessor (e.g., an embedded processor) for performing the operations.

Electronic devices according to various embodiments of the presentdisclosure may be smartphones, tablet personal computer (PC)s, mobilephones, video phones, e-book readers, desktop PCs, laptop computers,netbook computers, workstations, servers, personal digital assistants(PDAs), portable multimedia players (PMPs), MP3 players, medicaldevices, cameras, wearable devices, etc. Wearable devices may includeaccessory-type devices (e.g., watches, rings, bracelets, anklets,necklaces, glasses, contact lenses, or head-mounted devices (HMD)),fabric- or clothes-integrated devices (e.g., electronic clothes), bodyattaching-type devices (e.g., skin pads or tattoos), or body implantabledevices.

According to another embodiment, the electronic devices may be homeappliance such as televisions, digital video disk (DVD) players, audioplayers, refrigerators, air conditioners, cleaners, ovens, microwaveovens, washers, dryers, air cleaners, set-top boxes, home automationcontrol panels, security control panels, TV set-top boxes (e.g., SamsungHomeSync™, Apple TV™, or Google TV™), gaming consoles (e.g. Xbox™,PlayStation™), electronic dictionaries, electronic keys, camcorder,electronic picture frames, etc.

According to another embodiment, the electronic devices may be medicaldevices (e.g., various portable medical measuring devices such as bloodsugar measuring devices, heartbeat measuring devices, body temperaturemeasuring devices, etc., magnetic resource angiography (MRA) devices,magnetic resource imaging (MRI) devices, computed tomography (CT)devices, other medical imaging devices, ultrasonic devices, etc.),navigation devices, global navigation satellite system (GNSS) receivers,event data recorders (EDRs), flight data recorders (FDRs), automotiveinfotainment devices, sailing electronic devices (e.g., sailingnavigation devices or gyro compasses), avionics, security devices,vehicular head units, industrial or home robots, drones, automaticteller machines (ATMs), point of sales (POS) devices, or internet ofthings (IoT) devices (e.g., light bulbs, various sensors, sprinklers,fire alarms, thermostats, street lights, toasters, fitness equipment,hot water tanks, heaters, boilers, etc.).

According to another embodiment of the disclosure, the electronicdevices may be at least one of part of a piece of furniture,building/structures, vehicles, electronic boards, electronic signaturereceiving devices, projectors, various measurement devices (e.g.,devices for measuring water, electricity, gas, or electromagneticwaves), etc.

According to embodiments of the present disclosure, the electronicdevice may be flexible and may be a combination of the above-enumeratedelectronic devices. The electronic device disclosed herein is notlimited to the above-listed embodiments. As used herein, the term “user”may refer a human or another device (e.g., an artificial intelligentelectronic device) using the electronic device.

Referring to FIG. 1A, according to an embodiment of the presentdisclosure, an electronic device 101 is included in a networkenvironment 100. The electronic device 101 may include a bus 110, aprocessor 120, a memory 130, an input/output interface 150, a display160, a communication interface 170, and a camera module 180. In someembodiments, the electronic device 101 may exclude at least one of thecomponents or may add another component.

The bus 110 may include a circuit for connecting the components 110 to180 with one another and transferring communications (e.g., controlmessages or data) between the components.

The processor 120 may be a central processing unit (CPU), an applicationprocessor (AP), and/or a communication processor (CP). The processor 120may control at least one of the other components of the electronicdevice 101. The processor 120 may also perform operations and dataprocessing related to the various functions of the electronic device101. The processor 120 may include a microprocessor or any suitable typeof processing circuitry, such as one or more general-purpose processors(e.g., ARM-based processors), a Digital Signal Processor (DSP), aProgrammable Logic Device (PLD), an Application-Specific IntegratedCircuit (ASIC), a Field-Programmable Gate Array (FPGA), a GraphicalProcessing Unit (GPU), a video card controller, etc. In addition, itwould be recognized that when a general purpose computer accesses codefor implementing the processing shown herein, the execution of the codetransforms the general purpose computer into a special purpose computerfor executing the processing shown herein. Certain of the functions andsteps provided in the Figures may be implemented in hardware, softwareor a combination of both and may be performed in whole or in part withinthe programmed instructions of a computer. No claim element herein is tobe construed under the provisions of 35 U.S.C. 112(f), unless theelement is expressly recited using the phrase “means for.” In addition,an artisan understands and appreciates that a “processor” or“microprocessor” may be hardware in the claimed disclosure. Under thebroadest reasonable interpretation, the appended claims are statutorysubject matter in compliance with 35 U.S.C. § 101.

The memory 130 may include a volatile and/or non-volatile memory. Forexample, the memory 130 may store commands or data related to at leastone other component of the electronic device 101. According to anembodiment of the present disclosure, the memory 130 may store softwareand/or a program 140.

The program 140 may include, e.g., a kernel 141, middleware 143, anapplication programming interface (API) 145, and/or an applicationprogram (or “application”) 147. At least a portion of the kernel 141,middleware 143, or API 145 may be part of an operating system (OS). Forexample, the kernel 141 may control or manage system resources (e.g.,the bus 110, processor 120, or a memory 130) used to perform operationsor functions implemented in other programs (e.g., the middleware 143,API 145, or application program 147). The kernel 141 may also provide aninterface that allows the middleware 143, the API 145, or theapplication 147 to access the individual components of the electronicdevice 101. The middleware 143 may function as a relay to allow the API145 or the application 147 to communicate data with the kernel 141, forexample. Further, the middleware 143 may process one or more taskrequests received from the application program 147 in order of priority.In other words, the middleware 143 may assign priorities for usingsystem resources (e.g., bus 110, processor 120, or memory 130) to theapplication programs 147 and process one or more task requests based onthose priorities. The API 145 is an interface allowing the application147 to control functions provided from the kernel 141 or the middleware143. For example, the API 133 may include interfaces or functions (e.g.,commands) for filing control, window control, image processing or textcontrol.

The input/output interface 150 may transfer commands or data input fromthe user or another external device to the component(s) of theelectronic device 101 or may output commands or data received from thecomponent(s) of the electronic device 101 to the user or the otherexternal device.

The display 160 may include, e.g., a liquid crystal display (LCD), alight emitting diode (LED) display, an organic light emitting diode(OLED) display, or a microelectromechanical systems (MEMS) display, oran electronic paper display. The display 160 may display, e.g., variouscontents (e.g., text, images, videos, icons, or symbols) to the user.The display 160 may include a touchscreen and may receive, e.g., touch,gesture, proximity or hovering inputs via an electronic pen or a portionof the user's body (e.g. a finger).

The communication interface 170 may set up communication between theelectronic device 101 and an external electronic device (e.g., the firstelectronic device 102, the second electronic device 104, or the server106). For example, the communication interface 170 may be connected withthe network 162 through wireless or wired communication to communicatewith the external electronic devices.

Wireless communication, denoted as element 164 of FIG. 1. may becellular communication that employs protocols such as long termevolution (LTE), long term evolution-advanced (LTE-A), code divisionmultiple access (CDMA), wideband code division multiple access (WCDMA),universal mobile telecommunication system (UMTS), wireless broadband(WiBro), global system for mobile communication (GSM), etc. The wirelesscommunication may also use non-cellular protocols such aswireless-fidelity (Wi-Fi), light-fidelity (Li-Fi), Bluetooth, Bluetoothlow power (BLE), Zigbee, near-field communication (NFC), magnetic securetransmission (MST), radio frequency (RF), or body area network (BAN).According to an embodiment of the present disclosure, the wirelesscommunication may include global navigation satellite system (GNSS). TheGNSS may be, e.g., global positioning system (GPS), global navigationsatellite system (Glonass), Beidou navigation satellite system(hereinafter, “Beidou”) or Galileo, or the European globalsatellite-based navigation system. Hereinafter, the terms “GPS” and the“GNSS” may be interchangeably used. Wired communication, on the otherhand, may employ protocols such as universal serial bus (USB), highdefinition multimedia interface (HDMI), recommended standard (RS)-232,power line communication (PLC), plain old telephone service (POTS), etc.The network 162 may include be telecommunication networks, e.g.,computer networks (e.g., local area network (LAN) or wide area network(WAN)), the Internet, or telephone networks.

The camera module 180 may include at least one image sensor. Forexample, the image sensor in the camera module 180 may convert lightreceived from the outside of the electronic device into an electricalsignal and output the electrical signal. The electrical signal may beoutput through the bus 110 to the processor 120 and processed by theprocessor 120 or stored in the memory 130. The camera module 180 mayinclude a pixel array made up of a plurality of pixels, and the pixelarray may include photodiodes that convert light from the outside intoan analog electrical signal. Meanwhile, the image sensor in the cameramodule 180 may include an analog-to-digital converter (ADC) thatconverts the analog electrical signal into a digital electrical signaland outputs the digital electrical signal. The image sensor in thecamera module 180 may include a circuit for scanning the pixel array.The image sensor in the camera module 180 may include an internalmemory. The image sensor may temporarily store the digital electricalsignal, i.e., data output from a pixel, in the internal memory andoutput the digital electrical signal to an external circuit (e.g., thebus 110, the processor 120, or the memory 130). The image sensor in thecamera module 180 may include an interface used for data input/outputand may output data to an external circuit according to the output speedof the interface.

The first and second external electronic devices 102 and 104 each may bea device of the same or a different type from the electronic device 101.According to an embodiment of the present disclosure, all or some ofoperations executed on the electronic device 101 may be executed onanother or multiple other electronic devices (e.g., the electronicdevices 102 and 104 or server 106). According to an embodiment of thepresent disclosure, when the electronic device 101 is set to perform oneor more functions or services, the electronic device 101 may perform thefunctions or services on its own. However, alternatively, instead ofexecuting the functions or services on its own, the electronic device101 may request another device (e.g., electronic devices 102 and 104 orserver 106) to perform at least some functions associated therewith. Theother electronic device (e.g., electronic devices 102 and 104 or server106) may execute the requested functions and transfer the result of theexecution to the electronic device 101. The electronic device 101 mayprovide the requested functions or services by processing the receivedresult. To that end, cloud computing, distributed computing, orclient-server computing techniques may be used, for example.

FIG. 1B is a concept view illustrating an electronic device according toan embodiment of the present disclosure.

As shown in FIG. 1B, the electronic device 101 may be implemented in theform of a robot. The electronic device 101 may include a head portion190 and a body portion 193. The head portion 190 may be disposed on thebody portion 193. The head portion 190 and the body portion 193, in anembodiment, may be implemented in the shape of a human being's head andbody, respectively. For example, the head portion 190 may include afront cover 161 corresponding to the shape of a human being's face. Theelectronic device 101 may include a display 160 disposed at a positioncorresponding to the front cover 161. The display 160 may include atouchscreen and may receive, e.g., touch, gesture, proximity or hoveringinputs via an electronic pen or a portion of the user's body. Forexample, the display 160 may be disposed inside the front cover 161, andin this case, the front cover 161 may be made up of a transparent orsemi-transparent material. Alternatively, the front cover 161 and thedisplay 160 may be implemented as a single hardware device, such thatthe front cover 161 can be said to be the screen. The front cover 161may indicate a direction of interacting with the user and may include atleast one or more various sensors for image sensing in a direction ofinteracting with the user, at least one or more microphones forobtaining a voice, and at least one or more hardware (H/W) orinstrumental structures that may be an instrumental eye structure or adisplay for outputting a screen, that enables display through atemporary instrumental change or light in the form of directions beingnot differentiated, and that may be directed to the user wheninteracting with the user.

The head portion 190 may further include a communication interface 170,a sensor 171, and a camera module 180. The communication interface 170may receive various data from an external transmitter and transmit datagenerated by the electronic device 101 to an external receiver.According to an embodiment of the present disclosure, the communicationinterface 170 may be implemented as a microphone, and in this case, thecommunication interface 170 may receive voice commands from the user.The communication interface 170 may also be implemented as a speaker inwhich case the communication interface 170 may output the data generatedby the electronic device 101 as sound.

The sensor 171 may measure various qualities of the external environmentsurrounding the electronic device 101. For example, the sensor 171 maydetect the user or another electronic device approaching the electronicdevice 101. The sensor 171 may generate proximity information accordingto the approach of the user or the other electronic device. The sensor171 may detect the user's approach based on a signal from anotherelectronic device that the user is using. The sensor 171 may also detectthe user's motion or position.

The camera module 180 may capture images of the external environment.The electronic device 101 may identify the user or the other electronicdevice using at least one image obtained through the camera module 180.The electronic device 101 may also identify the user's motion orrelative position of the other electronic device based on image(s)obtained through the camera module 180.

The driver 191 may include at least one motor to move the head portion190. The driver 191 may also be used to move the electronic device 101or vary the positions of the other components of the electronic device101. As such, the driver 191 may be capable of moving up, down, left, orright with respect to one or more axes. The power supply 192 may feedpower to the electronic device 101.

The processor 120 may obtain data from another electronic device throughthe communication interface 170 or the sensor 171. The processor 120 mayinclude at least one data analyzing module. The data analyzing modulemay process received data and transmit the processed data. For example,the processor 120 may analyze data corresponding to a voice inputreceived from the user, generate results of the analysis which needs tobe sent to another electronic device, and send the results to the otherelectronic device.

The memory 130 is storage for permanently or temporarily storinginformation related to functions or services of the electronic device101. The memory 130 may be present in the electronic device 101 or maybe present in a cloud or other server connected to the electronic device101 through a network. The memory 130 may store personal information foruser authentication, attribute-related information relating to themanner in which services are provided to the user, or relationshipinformation for various entities (users or other electronic devices)that may interact with the electronic device 101. Here, the relationshipinformation may be updated as the electronic device 101 is used or maybe otherwise changed. The processor 120 may functionally control thesensor 171, the input/output interface 150, the communication interface170, and the memory 130 to provide function or services to the user.Further, at least a portion of the processor 120 or the memory 130 mayinclude an information determining unit that may process informationobtained by the electronic device 101. For example, the informationdetermining unit may extract data required for a particular service fromthe information obtained by the sensor 171 or the communicationinterface 170.

The robot-type implementation of the electronic device 101 shown in thefigure is merely an example, and there are no limitations as to the typeof implementation. For example, the electronic device 101 may beimplemented as a standalone robot. The electronic device 101 may beimplemented as a docking station that holds a tablet PC or smart phone.Further, the electronic device 101 may be a stationary device or may bea mobile device. When it is a mobile device, the electronic device 101may achieve mobility using wheels, mechanical caterpillar devices,mechanical legs, etc. Or the electronic device 101 may be a flyingdrone.

FIG. 2A is a block diagram illustrating an electronic device 201according to an embodiment of the present disclosure. An electronicdevice 201 may include the whole or part of, e.g., the electronic device101 of FIG. 1. The electronic device 201 may include one or moreprocessors (e.g., application processors (APs)) 210, a communicationmodule 220, a subscriber identification module (SIM) 224, a memory 230,a sensor module 240, an input device 250, a display 260, an interface270, an audio module 280, a camera module 291, a power management module295, a battery 296, an indicator 297, and a motor 298.

The processor 210 may control multiple hardware and software componentsconnected to the processor 210 by running, e.g., an operating system orapplication programs. In doing so, and the processor 210 may process andcompute various pieces of data. The processor 210 may be implemented in,e.g., a system on chip (SoC). According to an embodiment of the presentdisclosure, the processor 210 may further include a graphic processingunit (GPU) and/or an image signal processor. The processor 210 mayinclude at least some (e.g., the cellular module 221) of the componentsshown in FIG. 2A. The processor 210 may loads commands or data receivedfrom at least one of the other components (e.g., the non-volatilememory) onto a volatile memory, process the commands or data, and storeresultant data in the non-volatile memory.

The communication module 220 may have the same or similar configurationto the communication interface 170 of FIG. 1A. The communication module220 may include, e.g., a cellular module 221, a wireless fidelity(Wi-Fi) module 223, a Bluetooth (BT) module 225, a GNSS module 227, aNFC module 228, and a RF module 229. The cellular module 221 may providevoice call, video call, text messaging, or Internet services through,e.g., a cellular communication network. According to an embodiment ofthe present disclosure, the cellular module 221 may performidentification or authentication of the electronic device 201 in thecommunication network using a subscriber identification module 224(e.g., the SIM card). The cellular module 221 may perform at least someof the functions provided by the processor 210. The cellular module 221may include a communication processor (CP). At least some (e.g., two ormore) of the cellular module 221, the Wi-Fi module 223, the Bluetoothmodule 225, the GNSS module 227, or the NFC module 228 may be includedin a single integrated circuit (IC) or an IC package. The RF module 229may transceive data using RF signals. The RF module 229 may include,e.g., a transceiver, a power amp module (PAM), a frequency filter, a lownoise amplifier (LNA), or an antenna. According to an embodiment of thepresent disclosure, at least one of the cellular module 221, the Wi-Fimodule 223, the Bluetooth module 225, the GNSS module 227, or the NFCmodule 228 may transceive RF signals through its own standalone RFmodule. The subscription identification module 224 may include, e.g., aremovable SIM card or an embedded SIM, and may contain uniqueidentification information (e.g., integrated circuit card identifier(ICCID) or subscriber information (e.g., an international mobilesubscriber identity (IMSI)).

The memory 230 (e.g., the memory 130) may include, e.g., an internalmemory 232 or an external memory 234. The internal memory 232 mayinclude at least one of, e.g., a volatile memory (e.g., a dynamic RAM(DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), etc.) ora non-volatile memory (e.g., a one-time programmable ROM (OTPROM), aprogrammable ROM (PROM), an erasable and programmable ROM (EPROM), anelectrically erasable and programmable ROM (EEPROM), a mask ROM, a flashROM, a flash memory (e.g., a NAND flash, or a NOR flash), a hard drive,or solid-state drive (SSD)). The external memory 234 may include a flashdrive (e.g., a compact flash (CF) memory, a secure digital (SD) memory,a micro-SD memory, a min-SD memory, an extreme digital (xD) memory, amulti-media card (MMC), or a Memory Stick™). The external memory 234 maybe functionally or physically connected with the electronic device 201via various interfaces.

The sensor module 240 may measure a physical quantity or detect a motionstate of the electronic device 201, and the sensor module 240 mayconvert the measured or detected information into an electrical signal.The sensor module 240 may include at least one of, e.g., a gesturesensor 240A, a gyro sensor 240B, an atmospheric pressure sensor 240C, amagnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, aproximity sensor 240G, a color sensor 240H (e.g., a red-green-blue (RGB)sensor, a bio sensor 240I, a temperature/humidity sensor 240J, anillumination sensor 240K, or an Ultra Violet (UV) sensor 240M.Additionally or alternatively, the sensing module 240 may include, e.g.,an e-nose sensor, an electromyography (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aninfrared (IR) sensor, an iris sensor, or a finger print sensor. Thesensor module 240 may further include a control circuit for controllingat least one or more of the sensors included in the sensing module.According to an embodiment of the present disclosure, the electronicdevice 201 may further include a processor configured to control thesensor module 240 as part of the processor 210 or separately from theprocessor 210, and the electronic device 2701 may control the sensormodule 240 while the processor 1210 is in a sleep mode.

The input unit 250 may include, e.g., a touch panel 252, a (digital) pensensor 254, a key 256, or an ultrasonic input device 258. The touchpanel 252 may use at least one of capacitive, resistive, infrared, orultrasonic methods. The touch panel 252 may further include a controlcircuit. The touch panel 252 may further include a tactile layer toprovide an immediate reaction to the user. The (digital) pen sensor 254may include, e.g., part of the touch panel or a separate sheet forrecognition. The key 256 may include e.g., a physical button, opticalkey or key pad. The ultrasonic input device 258 may sense an ultrasonicwave generated from an input tool through a microphone (e.g., themicrophone 288) to identify data corresponding to the sensed ultrasonicwave.

The display 260 (e.g., the display 160) may include a panel 262, ahologram device 264, a projector 266, and/or a control circuit forcontrolling the same. The panel 262 may be implemented to be flexible,transparent, or wearable. The panel 262, together with the touch panel252, may be configured in one or more modules. According to anembodiment of the present disclosure, the panel 262 may include apressure sensor (or force sensor) that may measure the strength of apressure by the user's touch. The pressure sensor may be implemented ina single body with the touch panel 252 or may be implemented in one ormore sensors separate from the touch panel 252. The hologram device 264may make three dimensional (3D) images (holograms) in the air by usinglight interference. The projector 266 may display an image by projectinglight onto a screen. The screen may be, for example, located inside oroutside of the electronic device 201.

The interface 270 may include e.g., a high definition multimediainterface (HDMI) 272, a USB 274, an optical interface 276, or aD-subminiature (D-sub) 278. The interface 270 may be included in forexample the communication interface 170 shown in FIG. 1. Additionally oralternatively, the interface 270 may include a mobile high-definitionlink (MHL) interface, a secure digital (SD) card/multimedia card (MMC)interface, or infrared data association (IrDA) standard interface.

The audio module 280 may convert, e.g., a sound signal into anelectrical signal and vice versa. At least a part of the audio module280 may be included in for example the input/output interface 145 asshown in FIG. 1. The audio module 280 may process sound informationinput or output through e.g., a speaker 282, a receiver 284, an earphone286, or a microphone 288.

The camera module 291 may be a device for capturing still images andvideos, and may include, according to an embodiment of the presentdisclosure, one or more image sensors (e.g., front and back sensors), alens, an image signal processor (ISP), or a flash such as an LED orxenon lamp. According to an embodiment of the present disclosure, thecamera module 291 may include the whole or part of the camera module180.

The power manager module 295 may manage power of the electronic device201, for example. According to an embodiment of the present disclosure,the power manager module 295 may include a power management Integratedcircuit (PMIC), a charger IC, or a battery or fuel gauge. The PMIC mayemploy a wired and/or wireless recharging scheme. The wireless chargingscheme may use for example, magnetic resonance charging, magneticinduction charging, or electromagnetic wave based charging. In thiscase, additional circuits, such as a coil loop, a resonance circuit, arectifier, or the like may be needed for wireless charging. The batterygauge may measure the remaining power left in the battery 296, or thevoltage, current, and temperature of the battery 296 while the battery296 is being charged. The battery 296 may be a rechargeable battery or asolar battery.

The indicator 297 may indicate a particular state of the electronicdevice 201 or a part (e.g., the processor 210) of the electronic device,including e.g., when the electronic device is booting, has received amessage, or is recharging. The motor 298 may convert an electric signalto a mechanical vibration and may generate a vibrational or hapticeffect. The electronic device 201 may include a mobile TV supportingdevice (e.g., a GPU) that may process media data as per, e.g., digitalmultimedia broadcasting (DMB), digital video broadcasting (DVB), orMediaFlo™ standards. Each of the aforementioned components of theelectronic device may include one or more parts, and the names of theparts may vary depending on the type of the electronic device. Accordingto various embodiments, the electronic device (e.g., the electronicdevice 201) may exclude some elements or include more elements, or someof the elements may be combined into a single component that may performthe same function as by the elements prior to the combination.

FIG. 2B is a block diagram illustrating an electronic device accordingto an embodiment of the present disclosure. As shown in FIG. 2b , theprocessor 210 may be connected with an image recognition module 241. Theprocessor 210 may also be connected with a behavior module 244. Theimage recognition module 241 may include a 2-dimensional (2D) camera 242and/or a depth camera 243. The image recognition module 241 may performrecognition based on the captured images and transfer the recognizedresult to the processor 210. The behavior module 244 may include afacial expression motor 245, a body pose motor 246, and/or a movingmotor 247. The processor 210 may control the facial expression motor245, the body pose motor 246, and/or the moving motor 247 to control themovement of the electronic device 101 when it is implemented as a robot.The electronic device 101 may include the elements shown in FIG. 2b inaddition to the elements shown in FIG. 2 a.

According to an embodiment of the present disclosure, an electronicdevice may comprise a camera, at least one motor, a communicationinterface, at least one processor, and a memory electrically connectedwith the processor, wherein the memory may store commands that, whenexecuted by the processor, cause the processor to identify a motion ofan external object using a first image obtained by controlling thecamera, obtain first direction information based on the identifiedmotion of the external object, drive the at least one motor so that thecamera faces a direction determined according to the first directioninformation, and identify a second electronic device from a second imageobtained by controlling the camera facing the determined direction.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto transmit data corresponding to the second image to at least oneserver through the communication interface, receive data correspondingto the second electronic device determined based on the datacorresponding to the second image from the at least one server, andidentify the second electronic device based on the received datacorresponding to the second electronic device.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto receive at least one voice input, transmit data corresponding to theat least one voice input to at least one server through thecommunication interface, receive data corresponding to the secondelectronic device determined based on the data corresponding to the atleast one voice input from the at least one server, and identify thesecond electronic device based on the received data corresponding to thesecond electronic device.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto, when the second electronic device is determined to be a controllableelectronic device, obtain data corresponding to the second electronicdevice from the memory, connect with the second electronic devicethrough the communication interface using the obtained data, and controlthe second electronic device.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto, when the data corresponding to the second electronic device is notin the memory, receive the data corresponding to the second electronicdevice from at least one server through the communication interface, andconnect with the second electronic device using the data received fromthe at least one server.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto store the obtained first direction information and the datacorresponding to the second electronic device, which is received fromthe at least one server, in the memory.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto receive at least one voice input, obtain a command related to controlof the second electronic device from the at least one voice input, andtransmit a signal corresponding to the obtained command to the secondelectronic device.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto identify a state change in the second electronic device using a thirdimage obtained by controlling the camera facing the direction, when theidentified state change in the second electronic device does notcorrespond to the obtained command, output feedback corresponding to theat least one voice input.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto output a request related to designation of a command corresponding tothe second electronic device, store the obtained first directioninformation and a command corresponding to the request in the memory,and upon receiving at least one voice input including the command storedin the memory, transmit a signal corresponding to the command to thesecond electronic device.

According to an embodiment of the present disclosure, the memory maystore commands that, when executed by the processor, cause the processorto, upon detecting a motion of the second electronic device, drive theat least one motor to trace the motion of the second electronic device,obtain second direction information about the second electronic devicebased on a result of tracing the motion of the second electronic device,and store the obtained second direction information in the memory.

FIG. 3 is a view illustrating a method for controlling an electronicdevice using another electronic device according to an embodiment of thepresent disclosure. According to an embodiment of the presentdisclosure, an electronic device 301 may include the whole or part ofthe electronic device 101 of FIGS. 1A and 1B.

Referring to FIG. 3, a user 310 of the electronic device 301 may controlan object 320, e.g., a smart lamp, through the electronic device 301.The electronic device 301 may obtain information related to the object320 using a camera (e.g., the camera module 180 or 291) or input/outputinterface (e.g., the input/output interface 150 of FIG. 1A or the inputdevice 250 of FIG. 2A) provided in the electronic device 301.

According to an embodiment of the present disclosure, the electronicdevice 301 may obtain at least one image corresponding to the user 310through the camera of the electronic device 301. The electronic device301 may detect a motion of the user 310 using the at least one imageobtained corresponding to the user 310. The electronic device 301 mayobtain at least one piece of direction information based on the detectedmotion of the user 310. For example, the electronic device 301 mayobtain at least one piece of direction information based on the motionof the user 310 pointing at least one of his or her body parts in aparticular direction. For example, the electronic device 301 may detectthe motion of the user 310 pointing his or her finger in a particulardirection and may obtain at least one piece of direction informationcorresponding thereto. In another example, the electronic device 301 maydetect the motion of the user 310 pointing his or her eyes or head in aparticular direction and may obtain at least one piece of directioninformation corresponding thereto.

Thus, the user 310 of the electronic device 301 may perform the motionof pointing to the object 320 which he or she desires to control, andthe electronic device 301 may detect the motion of the user 310 throughthe camera. The electronic device 301 may obtain at least one piece ofdirection information related to the object 320 based on the detectedmotion of the user 310. The electronic device 301 may determine theposition of the object 320 based on the at least one piece of directioninformation obtained. For example, the electronic device 301 maydetermine a relative position of the object 320 based on the relativepositions of a plurality of objects identified through the camera andthe direction information obtained. The electronic device 301 maydetermine an absolute position of the object 320 based on coordinateinformation about a preset reference point and the direction informationobtained.

According to an embodiment of the present disclosure, the electronicdevice 301 may obtain at least one image containing information relatedto the object 320 through the camera of the electronic device 301. Theelectronic device may identify the information related to the object 320in the obtained image. For example, the electronic device 301 maycapture an image by which the shape, model name, or position of theobject 320 may be identified through the camera of the electronic device301. Alternatively, the electronic device 301 may capture an image of abarcode or QR code that identifies the object 320. Thus, the electronicdevice 301 may identify information of the object 320, e.g., shape,model name, or position of the object 320, from the obtained image.

The electronic device 301 may receive at least one input containing theinformation related to the object 320 through an input/output interfaceprovided in the electronic device 301. The electronic device 301 mayidentify the information related to the object 320 from at least oneinput received. For example, the electronic device 301 may receive atleast one input containing the information for identifying the object320, such as the shape, model name, or position of the object 320,through the input/output interface of the electronic device 301 andextract this information. The electronic device 301 may be activatedwhen a predesignated voice input is received. For example, the activatedelectronic device 301 may receive a voice input 313 dictating “turn onthe lamp,” from the user 310. The electronic device 301 may recognizethat the object 320 corresponds to the received voice input 313. Theelectronic device 301 may output a voice dictating “Ok!” when the object320 is recognized.

The electronic device 301 may recognize the object 320 based oninformation obtained from other devices. For example, to recognize theobject 320, the electronic device 301 may transmit the obtainedinformation through a communication interface (e.g., the communicationinterface 170 of FIG. 1 or the communication module 220 of FIG. 2) toone or more servers. The electronic device 301 may then receive at leastone piece of information necessary to recognize the object 320 from theone or more servers. The electronic device 301 may then recognize theobject 320 based on the information received from the one or moreservers. For example, the electronic device 301 may receive images ofthe object 320 captured at various angles. The electronic device 301 mayrecognize the object 320 using these images.

The electronic device 301 may move at least part of the electronicdevice 301 or reposition the electronic device 301 through a driver(e.g., the driver 191) provided in the electronic device 301 to locatethe recognized object 320. For example, the driver may be provided in aportion inside or outside a head portion (e.g., the head portion 190) orbody portion (e.g., the body portion 193) of the electronic device ormay be provided where the head portion and the body portion connecttogether. The driver may include at least one motor. The electronicdevice 301 may activate at least one motor to move at least part of theelectronic device 301 or reposition the electronic device 301. Theelectronic device 301 may redirect the camera of the electronic device301 by moving the head portion (e.g., the head portion 190), whichincludes the camera. For example, the electronic device 301 may redirectthe camera of the electronic device 301 in a first direction based onthe direction information obtained from the user. The electronic device301 may obtain at least one image corresponding to the first directionand may locate the object 320 using the obtained image.

Where the object 320 is identified from the image corresponding to thefirst direction, the electronic device 301 may determine whether theidentified object 320 is positioned at the center of the image. If not,the electronic device 301 may reposition the camera (i.e. reposition thehead portion 190) so that the identified object 320 is at the center ofthe image obtained through the camera.

If the object 320 is not identified from the image corresponding to thefirst direction, the electronic device 301 may redirect the camera in asecond direction. The electronic device 301 may obtain at least oneimage corresponding to the second direction through the camera and maylocate the object 320 using the image in the second direction. Theelectronic device 301 may repeatedly redirect the camera until theobject 320 is identified.

According to an embodiment of the present disclosure, the electronicdevice 301, after locating the object 320 through the camera, may outputfeedback content 303 in the direction of the object. For example, wherethe object 320 is identified from at least one image obtained throughthe camera, the electronic device 301 may output light using aflashlight or its display, a sound using a speaker, or a vibration usinga vibration motor.

The electronic device 301 may generate a control signal corresponding toat least one motion of the user 310 and a voice input received from theuser 310. The electronic device 301 may transmit the generated controlsignal through the communication interface of the electronic device 301to the object 320. Upon receiving the control signal generated by theelectronic device 301, the object 320 may perform the function requiredby the received control signal. For example, a lamp, upon receipt of acontrol signal corresponding to “turn on the lamp,” may turn on.

The electronic device 301 may identify that the object 320 has performedthe function required by the control signal through the camera. Forexample, the electronic device 301, which has sent a control signalcorresponding to “turn on the lamp” to the lamp 320, may identifywhether the lamp 320 turns on through the camera. For example, theelectronic device 301 may obtain at least one image containing the lamp320 through a camera that faces the lamp 320. The electronic device 301may then identify whether the lamp 320 has turned on by analyzing theobtained image.

FIG. 4 is a flowchart illustrating a method for enrolling an object tobe controlled using an electronic device according to an embodiment ofthe present disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), and a processor (e.g.,the processor 120 or 210). Among the components of the electronicdevice, the processor may perform the method.

Referring to FIG. 4, the processor may obtain information related to anobject through the camera or the input/output interface in operation410. For example, the processor may control the camera to obtain atleast one image of the user. The processor may detect the user's motionusing the obtained image. The processor may obtain at least one piece ofdirection information related to the object based on the detected usermotion. The processor may recognize the object based on the at least onepiece of direction information obtained.

The processor may control the camera to obtain at least one imageincluding object-related information. The processor may identify theobject-related information from the at least one image obtained. Theprocessor may recognize the object based on the identified information.For example, the processor may identify information for identifying theobject, e.g., the shape, model name, or position of the object, from theobtained image. The processor may recognize the object by comparing theidentified information with information stored in the memory of theelectronic device or a database of one or more servers.

The processor may receive at least one input including object-relatedinformation through the input/output interface. The processor mayidentify the object-related information from the input received. Theprocessor may recognize the object based on the identified information.For example, the processor may identify information for identifying theobject, e.g., the shape, model name, or position of the object, from theinput received. The processor may recognize the object by comparing theidentified information with information stored in the memory of theelectronic device or a database of one or more servers.

For example in operation 420, the processor may activate at least onemotor to locate the object through the camera. In particular, theprocessor may recognize the object based on information obtained throughthe camera or input/output interface. The processor may use the at leastone piece of direction information obtained in step 410 and may activateat least one motor provided in the electronic device based on the atleast one piece of direction information obtained in step 410. As such,the camera may be redirected in a direction corresponding to the atleast one piece of direction information obtained in step 410. Theprocessor may control the redirected camera to obtain at least oneimage, and the processor may locate the recognized object using at leastone image obtained.

In operation 430, the processor may determine whether the object locatedthrough the camera is a controllable object. For example, to determinewhether the object is controllable, the processor may identify whetherthe object located through the camera is included in a list ofcontrollable objects that is stored in the memory of the electronicdevice or a database of one or more servers. Alternatively, theprocessor may attempt to connect to the object through the communicationinterface of the electronic device to determine whether the object is acontrollable one.

Upon determining that the object is uncontrollable, the processor mayperform operation 490. In operation 490, the processor may outputcontent related to the object being uncontrollable. For example, theprocessor may output a voice speaking, “this object cannot becontrolled,” through the speaker of the electronic device. According toanother embodiment, the processor may display the content insteadthrough the display of the electronic device. According to an embodimentof the present disclosure, the processor may confirm whether the objectis the object that the user intends to control by outputting a voicespeaking “do you want to control this object?”

Upon determining that the object is controllable, the processor mayperform operation 440. In operation 440, the processor may determinewhether the object located through the camera is one already enrolled(i.e., registered as a device that can be controlled by the electronicdevice). For example, the processor may identify information stored inthe memory of the electronic device or a database of one or moredatabases to determine whether the object is one already enrolled.

If the object is determined not to have been enrolled yet, the processormay perform operation 450. In operation 450, the processor may receive arequest for newly enrolling the object from the user and store datacorresponding to the object in the memory. For example, the processormay output content inquiring whether to newly enroll the object. Forexample, the processor may output a voice speaking, “do you want toenroll this object? Upon receiving a request for new enrollment of theobject from the user, the processor may obtain data necessary toestablish connection with the object from one or more servers and storethe obtained data in the memory of the electronic device. Also, uponreceiving a request for new enrollment of the object from the user, theprocessor may store position information about the object obtainedthrough the camera or the input/output interface in the memory of theelectronic device.

When the object is determined to have already been enrolled or newenrollment is accomplished, the processor may perform operation 460. Inoperation 460, the processor may establish a connection with the objectthrough the communication interface of the electronic device.

In operation 470, the processor may send a request related todesignation of a command corresponding to the connected object and storethe command in the memory of the electronic device. For example, wherethe connection with the object is established through the communicationinterface, the processor may output content inquiring whether todesignate command(s) corresponding to the connected object. For example,the processor may output a voice speaking, “do you want to set a commandcorresponding to this object?” The processor may receive at least onecommand corresponding to the connected object from the user and storethe received command in the memory of the electronic device or adatabase of the one or more servers.

In operation 480, upon receipt of a command-containing input, theprocessor may transmit a signal corresponding to the received command tothe object. For example, upon receiving an input that corresponds to acommand stored in the memory of the electronic device or a database ofthe one or more servers, the processor may generate a signalcorresponding to the stored command. The processor may transmit thegenerated signal through the interface to the object. For example, theprocessor may receive a voice input speaking, “turn on the lamp,”generate a signal corresponding thereto, and transmitting the signalthrough the communication interface to the object. Upon receipt of thesignal, the object, e.g., lamp, may turn on.

Operations 410 to 490 are provided for the purposes of describingembodiments of the present disclosure, and operations 410 to 490 are notnecessarily performed sequentially nor are all of the operationsnecessary. For example, only some of the operations may be performed.

FIG. 5A is a flowchart illustrating a method for controlling at leastone object using an electronic device according to an embodiment of thepresent disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), and a processor (e.g.,the processor 120 or 210). Among the components of the electronicdevice, the processor may perform the method.

Referring to FIG. 5A, the processor may receive a command to control anobject through the camera or input/output interface in operation 500.For example, the processor may control the camera to obtain at least oneimage of the user. The processor may detect the user's motion using theobtained image. The processor may recognize an object to be controlledbased on the detected user motion. For example, the processor may detectthe user's motion of pointing her finger to the object. The processormay also recognize the object based on the user's voice input that isreceived through the input/output interface. For example, correspondingto receiving a sound input speaking “turn on the lamp on your left,” theprocessor may recognize the lamp relatively positioned on the left ofthe electronic device is the object to be controlled.

In operation 505, the processor may determine whether a connection withthe object is established and obtain data corresponding to the objectfrom the memory. The processor may determine whether a connection withthe recognized object has been established. Upon determining that noconnection is established with the recognized object, the processor mayobtain information necessary to establish a connection with therecognized object from the memory of the electronic device or a databaseof one or more servers. The processor may then establish a connectionwith the recognized object based on the obtained information. Upondetermining that a connection has been established with the recognizedobject, the processor may obtain data corresponding to the recognizedobject from the memory of the electronic device or a database of one ormore servers.

In operation 510, the processor may obtain direction information aboutthe recognized object based on at least one of data items obtained inoperation 505 and the command received in operation 500. For example,the processor may obtain the direction information about the recognizedobject based on the user's motion that is obtained via the camera.Alternatively, the processor may obtain the direction information aboutthe recognized object based on data obtained from the memory of theelectronic device or database of one or more servers.

In operation 515, the processor may activate at least one motor so thatthe camera faces the direction determined by the obtained directioninformation. For example, the camera of the electronic device may beprovided in a portion of the electronic device, e.g., a head portion(e.g., the head portion 190) of the electronic device. The processor mayactivate at least one motor included in a driver (e.g., the driver 191)of the electronic device to move the head portion of the electronicdevice. As the head portion of the electronic device moves, the cameraof the electronic device may be redirected. According to an embodimentof the present disclosure, the obtained direction information mayinclude information corresponding to the position of the recognizedobject. The processor may drive at least one motor to allow the camerato face the direction determined by the obtained direction informationso that the camera of the electronic device faces the recognized object.

In operation 520, the processor may determine whether the recognizedobject is an external object that is present outside the space where theelectronic device is positioned. For example, when the recognized objectis not in the space where the electronic device is positioned, theprocessor may have difficulty in locating the object through the camera.When the recognized object is outside the space where the electronicdevice is positioned, the processor may perform operation 540.

If the recognized object is within the space where the electronic deviceis positioned, the processor may perform operation 525. In operation525, the processor may determine whether the recognized object isidentified through the camera facing the determined direction. Forexample, the processor may control the camera to obtain at least oneimage in the determined direction. The processor may then identify atleast one object in the at least one image. The processor may determinewhether the at least one object identified is the same as the recognizedobject. When the recognized object is identified through the camerafacing the determined direction, the processor may perform operation540.

But if the recognized object is not identified through the camera facingthe determined direction, the processor may perform operation 530. Inoperation 530, the processor may drive at least one motor to locate therecognized object. For example, the processor may drive at least onemotor to redirect the camera.

In operation 535, the processor may determine whether the recognizedobject is identified through the camera facing the changed direction.For example, the present disclosure may control the camera facing thechanged direction to obtain at least one image in the changed direction.The processor may then identify at least one object in the at least oneimage. The processor may determine whether the at least one objectidentified is the same as the recognized object. According to anembodiment of the present disclosure, operations 530 through 535 may berepeated until the recognized object is identified or until scanning ofthe space where the electronic device is positioned is complete.

If the recognized object is not identified through the camera facing inthe changed direction, the processor may perform operation 545. Inoperation 545, the processor may enroll the recognized object as anexternal object which is present outside the space where the electronicdevice is positioned, and the processor may update the datacorresponding to the object. For example, upon failing to identify therecognized object despite omni-directional scanning of the space wherethe electronic device is positioned using the camera, the processor maydetermine that the recognized object is an external object which ispresent outside the space where the electronic device is positioned.Based on this determination, the processor may update the position datastored in the memory of the electronic device or database of one or moreservers.

When the recognized object is identified through the camera facing thechanged direction the processor may perform operation 540. In operation540, the processor may output content corresponding to a receivedcommand, with the camera positioned to face the determined or changeddirection. For example, when operation 540 is performed after operation520, the processor may abstain from driving at least one motor and onlyoutput content corresponding to the received command. In this case, thecontent corresponding to the received command may be content outputcorresponding to transmission, to the recognized object, of a signalcorresponding to the received command. For example, the processor mayoutput visual content using the flashlight or display of the electronicdevice, sound using the speaker of the electronic device, or vibrationusing a vibration motor of the electronic device.

In another example, when operation 540 is performed after operation 525,the processor may output the content corresponding to the receivedcommand, with the camera positioned to face the determined direction.Where operation 540 is performed after operation 535, the processor mayoutput the content corresponding to the received command, with thecamera positioned to face the changed direction. In this case, thecontent corresponding to the received command may be feedback contentoutput in the direction of the recognized object after the recognizedobject has been located through the camera. For example, the processormay output visual content using the flashlight or display of theelectronic device, sound using the speaker of the electronic device, orvibration using a vibration motor of the electronic device.

FIG. 5B is a flowchart illustrating a method for identifying a result ofcontrolling at least one object using an electronic device according toan embodiment of the present disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), and a processor (e.g.,the processor 120 or 210). Among the components of the electronicdevice, the processor may perform the method.

According to an embodiment of the present disclosure, the processor ofthe electronic device may perform the operations of FIG. 5B afterperforming the operations of FIG. 5A. Upon receipt of a command forcontrolling an object through the camera or input/output interface, theprocessor of the electronic device may perform the operations of FIG.5B.

Referring to FIG. 5B, the processor may transmit a signal correspondingto the received command to the object in operation 550. For example, theprocessor may receive a command to control the object through the cameraor input/output interface. The processor may generate a signalcorresponding to the received command and transmit the generated signalthrough the communication interface to the object.

In operation 555, the processor may determine whether the object is anexternal object that is present outside the space where the electronicdevice is positioned. For example, when the object is present outsidethe space where the electronic device is positioned, the processor mayhave difficulty in identifying state changes in the object through thecamera. When the object is present outside the space where theelectronic device is positioned, the processor may perform no additionaloperation.

But if the object is within the space where the electronic device ispositioned, the processor may perform operation 560. In operation 560,the processor may control the camera facing towards the object to obtainat least one image. For example, upon receiving a command to control theobject, the processor may obtain direction information about the object.The processor may obtain the direction information about the objectbased on the user's motion and the data obtained from the memory of theelectronic device or database of one or more servers. The processor maydetermine the position of the object according to the obtained directioninformation and drive at least one motor of the electronic device basedon a result of the determination to redirect the camera of theelectronic device. By so doing, the processor may control the camera toface the object.

In operation 565, the processor may identify a state change in theobject using at least one image obtained. For example, the presentdisclosure may control the camera facing towards the object to obtain atleast one image. By analyzing the at least one image obtained, theprocessor may identify a state change in the object. For example, theprocessor may identify that the object has turned on in response to a“turn on” command.

In operation 570, the processor may determine whether the identifiedstate change in object is a state change corresponding to the receivedcommand. The processor may compare an image of the object obtainedbefore transmitting the signal corresponding to the received commandwith an image of the object obtained afterwards. For example, theprocessor may receive a voice input speaking, “turn on the lamp.” Theprocessor may identify that the object, i.e. the lamp, is off beforesending the signal. The processor may also identify that the objectturned on after sending the signal. In this way, the processor maydetermine whether the identified state change in the object is a statechange corresponding to the received command.

If the identified state change in the object is not a state changecorresponding to the received command, the processor may performoperation 575. In operation 575, the processor may output contentrelated to a control failure. For example, the processor may output avoice speaking, “failed to control the object,” through the speaker ofthe electronic device. Alternatively, the processor may display contentrelated to the failure to control through the display of the electronicdevice.

In operation 580, the processor may re-receive a command to control theobject through the camera or input/output interface. For example, afteroutputting the content related to the failure to control, the processormay re-receive a command to control the object from the user. Theprocessor may re-perform at least one of operations 550 to 580 based onthe re-received command.

FIG. 6 is a flowchart illustrating a method for identifying an objectthat is repositioned using an electronic device according to anembodiment of the present disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), and a processor (e.g.,the processor 120 or 210). Among the components of the electronicdevice, the processor may perform the method.

Referring to FIG. 6, the processor may detect a motion of an enrolledobject in operation 610. For example, the processor may identify atleast one object enrolled in the electronic device as an object to becontrolled, using at least one image obtained by the camera. Accordingto an embodiment of the present disclosure, to identify the positions orstates of the enrolled objects, the processor may drive at least onemotor provided in the electronic device to periodically change thedirection in which the camera of the electronic device faces. Theprocessor may then identify the positions or states of the enrolledobjects using the images obtained by the camera pointing in variousdirections.

The processor may detect the motion of one or more of the enrolledobjects by identifying, periodically or in real-time through the camera,the positions or states of the enrolled objects.

In operation 620, the processor may drive at least one motor to allowthe camera to face a moving object. The processor may, in real-time,trace the moving object or identify the repositioned object based onimages obtained by the camera. Alternatively, the processor may trackthe moving object using a sensor, e.g., a motion sensor or distancemeasuring sensor, provided in the electronic device. The processor maydrive at least one motor provided in the electronic device so that thecamera faces the moving object or the repositioned object.

In operation 630, the processor may determine whether the object isidentified through the camera. For example, the processor may analyze atleast one image obtained by the camera facing and determine whether themoving object or repositioned object is identified.

If the object is not identified through the camera, the processor mayperform operation 670. In operation 670, the processor may obtaindirection information corresponding to the last position where theobject is identified. The position of the object may have beenpreviously determined. However, due to its movement, the processor canno longer locate the object. For example, the object may have moved tobe outside the space where the electronic device is positioned, suchthat the object may not be identified through the camera of theelectronic device. The processor may enroll the object, which is nolonger identified through the camera, as an external object. Theprocessor may update pre-stored object position data based on directioninformation corresponding to the last position where the object isidentified.

When the object is identified through the camera, the processor mayperform operation 640. In operation 640, the processor may detect amotion of the identified object. For example, upon detecting thecontinuous motion of the identified object, the processor may re-drivethe at least one motor to trace the moving object. The processor mayrepeat operations 620 to 640 until the object stops moving or is notidentified any longer.

If the motion of the object is not detected, the processor may performoperation 650. In operation 650, the processor may obtain directioninformation corresponding to the current position where the object hasstopped moving. For example, the processor may obtain the directioninformation corresponding to the current position of the object based onthe degree at which the head portion (e.g., the head portion 190) of theelectronic device has turned up or down or left or right. The processormay obtain the direction information corresponding to the currentposition of the object based on relative positions of one or moreobjects identified through the camera.

According to an embodiment of the present disclosure, the processor maygenerate a map corresponding to the space where the electronic device ispositioned using the camera and at least one sensor (e.g., the sensormodule 240) provided in the electronic device. The map may be generatedusing a 360-degree panoramic image captured of the space where theelectronic device is positioned. The processor may map at least one setof coordinates to the generated map and may represent the positions ofthe objects identified in the map using the mapped set of coordinates.The processor may identify the current position of the object that hasstopped moving on the generated map, and the processor may obtain thedirection information corresponding to the identified position.

In operation 660, the processor may update data corresponding to theobject, which is stored in the memory, based on the obtained directioninformation. For example, the processor may update the position data ofthe object based on the obtained direction information. Upon receiving acommand to control the object after the update, the processor may driveat least one motor to allow the camera of the electronic device to facethe direction corresponding to the updated position.

According to an embodiment of the present disclosure, the position ofthe electronic device (e.g., the electronic device 101 or 201) may bevaried. For example, the electronic device may be repositioned by theuser. The electronic device may be moved using at least one motorprovided in the electronic device. Upon determining that the electronicdevice has been repositioned, the processor may use the camera, at leastone sensor, input/output interface, or communication interface toidentify the changed position.

For example, the processor may obtain at least one image by controllingthe camera when its position has changed. The processor may identify thechanged position of the electronic device based on the at least oneimage obtained. For example, the processor may compare at least oneimage obtained before the electronic device is repositioned with atleast one image obtained after the electronic device is repositioned andidentify the changed position by comparing the images. In doing so, theprocessor may identify an object that is commonly included in the imageobtained before the electronic device is repositioned and the imageobtained after the electronic device is repositioned, and the processormay identify the changed position of the electronic device based on theabsolute or relative position of the identified object. Upon failing toidentify an object commonly included in the image obtained before theelectronic device is repositioned and the image obtained after theelectronic device is repositioned, the processor may determine that thespace where the electronic device is positioned has changed.

The processor may identify the changed position of the electronic devicebased on information received through at least one sensor, input/outputinterface, or communication interface. For example, the processor mayobtain information about the changed position from the user through theinput/output interface. Or, the processor may obtain the informationabout the changed position through the communication interface, e.g.,the global positioning system (GPS). The processor may identify thechanged position of the electronic device based on the informationobtained through the input/output interface and the communicationinterface.

When the changed position of the electronic device is identified, theprocessor may identify one or more objects positioned around theelectronic device through the camera of the electronic device. To thatend, the processor may redirect the camera by using at least one motorof the electronic device. The processor may also generate a new mapcorresponding to the changed position. The map may be generated using a360-degree panoramic image captured of the space where the electronicdevice is positioned. The processor may map at least one set ofcoordinates to the generated map and may represent the positions of theobjects identified in the map using the mapped set of coordinates. Theprocessor may identify the current positions of one or more objectswhose relative positions have been changed due to the motion of theelectronic device based on the generated map, and the processor mayobtain direction information corresponding to the identified positions.

FIG. 7 are views illustrating a method for driving at least part of anelectronic device according to an embodiment of the present disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), a sensor (e.g., thesensor 171), and a processor (e.g., the processor 120 or 210). Among thecomponents of the electronic device, the processor may perform themethod.

FIG. 7A is a plan view of a particular space 710 where the electronicdevice 701 and an object 720 to be controlled are positioned. Forexample, the particular space 710 may mean a living room or bed room ina home.

According to an embodiment of the present disclosure, the processor maymove at least part of the electronic device 701 using at least one motorprovided in the electronic device 701. For example, a head portion(e.g., the head portion 190) of the electronic device 701 may include atleast one of a camera and a sensor. The processor may drive at least onemotor to move the head portion of the electronic device 701 to changethe direction of the camera.

For example, the processor may receive a command to control the object720 from the user. To identify the object 720 to be controlled throughthe camera, the processor may change the direction of the camera so thatit faces the object 720. The processor may drive at least one motor toturn the head portion of the electronic device, thereby allowing thecamera of the electronic device to face the object 720.

FIG. 7B illustrates a map 711 generated through the camera of theelectronic device 701. The processor may generate the map 711 for thespace 710 where the electronic device 701 is positioned using imagesobtained by the camera. The processor may define the map 711 using atleast one reference point and coordinates corresponding to the referencepoint. The processor may identify the object 720 in the map 711, and theprocessor may identify the position of the identified object 720 basedon the coordinates. The processor may store the map 711 and coordinatedata corresponding to the map 711 in the memory of the electronic deviceor a database of one or more servers. Upon receipt of a command tocontrol the object 720, the processor may identify the coordinates ofthe object 720 in the map 711, and the processor may drive at least onemotor to allow the camera of the electronic device 701 to face thedirection corresponding to the identified coordinates.

FIG. 7C illustrates an overall shape of the map 711 corresponding to thespace 710 where the electronic device 701 is positioned. For example,the processor may configure the overall shape of the map 711 as asphere. The processor may obtain at least one image corresponding to atleast part of the space 710 by controlling the camera of the electronicdevice. The processor may identify the object 720 from the at least oneimage obtained. The map 711 corresponding to the space 710 as shown inFIG. 7C may be defined with absolute coordinates using the sphericalcoordinate system or relative coordinates with respect to the electronicdevice 701.

FIG. 8 is a flowchart illustrating a method for controlling anelectronic device according to an embodiment of the present disclosure.

An electronic device (e.g., the electronic device 101 or 201) forperforming this method may include a camera (e.g., the camera module 180or 291), an input/output interface (e.g., the input/output interface 150of FIG. 1A or the input device 250 of FIG. 2A), at least one motor, acommunication interface (e.g., the communication interface 170 of FIG.1A or the communication module 220 of FIG. 2A), and a processor (e.g.,the processor 120 or 210). Among the components of the electronicdevice, the processor may perform the method.

Referring to FIG. 8, the processor may identify a motion of an externalobject using a first image obtained by controlling the camera inoperation 810. Here, the first image may mean a single or multipleimages obtained by the camera. For example, the processor may obtain aplurality of images corresponding to the external object. The processormay identify a motion of the external object using the plurality ofimages obtained. For example, the processor may identify the motion ofthe external object between a first time and a second time by comparingan image obtained at the first time with an image obtained at the secondtime.

In operation 820, the processor may obtain at least one piece ofdirection information based on the identified motion of the externalobject. For example, the processor may control the camera of theelectronic device to obtain at least one image corresponding to theexternal object. Here, the external object may be the user of theelectronic device. The processor may detect the user's motion using atleast one image obtained corresponding to the user. The processor mayobtain at least one piece of direction information based on the user'smotion detected. For example, the processor may obtain at least onepiece of direction information based on the user's motion pointing atleast one body part in a particular direction. The processor may detectthe user's motion pointing her finger in a particular direction and mayobtain at least one piece of direction information correspondingthereto. The processor may detect the user's motion pointing her head oreyes in a particular direction and may obtain at least one piece ofdirection information corresponding thereto.

For example, the user may make the motion of pointing at least one bodypart to a second electronic device that the user wishes to control, andthe processor may detect the user's motion through the camera. Theprocessor may obtain at least one piece of direction information relatedto the second electronic device based on the detected user motion. Theprocessor may determine the position of the second electronic devicebased on at least one piece of direction information obtained. Forexample, the processor may determine a relative position of the secondelectronic device based on a relative relationship in position among aplurality of objects identified through the camera and the at least onepiece of direction information obtained. The processor may determine anabsolute position of the second electronic device based on coordinateinformation about a preset reference point and the at least one piece ofdirection information obtained.

In operation 830, the processor may activate at least one motor so thatthe camera faces in the direction determined by the at least one pieceof direction information obtained. For example, the processor may changethe direction in which the camera of the electronic device faces bymoving the head portion (e.g., the head portion 190) including thecamera of the electronic device. The processor may change the directionin which the camera of the electronic device faces based on firstdirection information obtained in relation to the second electronicdevice. The processor may drive at least one motor to allow the cameraof the electronic device to face in the direction determined as per theobtained first direction information.

In operation 840, the processor may identify the second electronicdevice from a second image obtained by controlling the camera facing inthe determined direction. Here, the second image may mean a single ormultiple images obtained through the camera. For example, the processormay identify the second electronic device from at least one imageobtained by controlling the camera. When the second electronic device isidentified from at least one image corresponding to the determineddirection, the processor may determine whether the identified secondelectronic device is positioned at the center of the image obtainedthrough the camera. The processor may adjust the direction in which thecamera faces to place the identified second electronic device at thecenter of the image obtained through the camera.

If the second electronic device is not identified from at least oneimage corresponding to the determined direction, the processor maychange the direction in which the camera faces. The processor may obtainat least one image corresponding to the changed direction by controllingthe camera, and the processor may locate the second electronic devicefrom at least one image obtained. The processor may repeat theoperations of changing the direction in which the camera faces,obtaining an image corresponding to the changed direction, and locatingthe second electronic device using the obtained image until the secondelectronic device is identified.

FIG. 9 is a block diagram illustrating a program module according to anembodiment of the present disclosure. According to an embodiment of thepresent disclosure, the program module 910 (e.g., the program 140) mayinclude an operating system (OS) controlling resources related to theelectronic device (e.g., the electronic device 101) and/or variousapplications (e.g., the application 147) driven on the operating system.The operating system may include, e.g., Android™, iOS™, Windows™,Symbian™, Tizen™, or Bada™.

Referring to FIG. 9, the program module 910 may include a kernel 920(e.g., the kernel 141), middleware 930 (e.g., the middleware 143), anAPI 960 (e.g., the API 145), and/or an application 970 (e.g., theapplication program 147). At least a part of the program module 910 maybe preloaded on the electronic device or may be downloaded from anexternal electronic device (e.g., the electronic devices 102 and 104 orserver 106).

The kernel 920 may include, e.g., a system resource manager 921 or adevice driver 923. The system resource manager 921 may perform control,allocation, or recovery of system resources. According to an embodimentof the present disclosure, the system resource manager 921 may include aprocess managing unit, a memory managing unit, or a file system managingunit. The device driver 923 may include, e.g., a display driver, acamera driver, a Bluetooth driver, a shared memory driver, a USB driver,a keypad driver, a Wi-Fi driver, an audio driver, or an inter-processcommunication (IPC) driver.

The middleware 930 may provide various functions to the application 970through the API 960 so that the application 970 may use limited systemresources in the electronic device or provide functions jointly requiredby applications 970. According to an embodiment of the presentdisclosure, the middleware 930 may include at least one of a runtimelibrary 935, an application manager 941, a window manager 942, amultimedia manager 943, a resource manager 944, a power manager 945, adatabase manager 946, a package manager 947, a connectivity manager 948,a notification manager 949, a location manager 950, a graphic manager951, or a security manager 952.

The runtime library 935 may include a library module used by a compilerin order to add new functions while, e.g., the application 970 is beingexecuted. The runtime library 935 may perform input/output management,memory management, or arithmetic function processing.

The application manager 941 may manage the life cycle of, e.g., theapplications 970. The window manager 942 may manage GUI resources usedon the screen. The multimedia manager 943 may detect a format used toreproduce various media files and may encode or decode a media filethrough a codec appropriate for the relevant format. The resourcemanager 944 may manage the source code or memory space of theapplication 970. The power manager 945 may manage, e.g., the capacity,temperature, or power of the battery and determine and provide powerinformation necessary for the operation of the electronic device using acorresponding piece of information of such. According to an embodimentof the present disclosure, the power manager 945 may interwork with abasic input/output system (BIOS). The database manager 946 may generate,search, or vary a database to be used in the applications 970. Thepackage manager 947 may manage installation or update of an applicationthat is distributed in the form of a package file. The connectivitymanager 948 may manage, e.g., wireless connectivity. The notificationmanager 949 may provide an event, e.g., arrival message, appointment, orproximity alert, to the user. The location manager 950 may manage, e.g.,locational information on the electronic device. The graphic manager 951may manage, e.g., graphic effects to be offered to the user and theirrelated user interface. The security manager 952 may provide systemsecurity or user authentication, for example. According to an embodimentof the present disclosure, the middleware 930 may include a telephonymanager for managing the voice or video call function of the electronicdevice or a middleware module able to form a combination of thefunctions of the above-described elements. According to an embodiment ofthe present disclosure, the middleware 930 may provide a modulespecified according to the type of the operating system. The middleware930 may dynamically omit some existing components or add new components.

The API 960 may be a set of, e.g., API programming functions and mayhave different configurations depending on operating systems. Forexample, in the case of Android or iOS, one API set may be provided perplatform, and in the case of Tizen, two or more API sets may be offeredper platform.

The application 970 may include an application that may provide, e.g., ahome 971, a dialer 972, an SMS/MMS 973, an instant message (IM) 974, abrowser 975, a camera 976, an alarm 977, a contact 978, a voice dial979, an email 980, a calendar 981, a media player 982, an album 983, ora clock 984. The application 970 may also include a healthcareapplication (e.g., for measuring the degree of workout or blood sugar),or an application that provides environmental information (e.g., airpressure, moisture, or temperature information). According to anembodiment of the present disclosure, the application 970 may include aninformation exchanging application supporting information exchangebetween the electronic device and an external electronic device.Examples of the information exchange application may include, but is notlimited to, a notification relay application for transferring specificinformation to the external electronic device, or a device managementapplication for managing the external electronic device. For example,the notification relay application may transfer notification informationgenerated by applications of the electronic device to the externalelectronic device or receive notification information from the externalelectronic device and provide the received notification information tothe user. The device management application may install, delete, orupdate functions (e.g., turn-on/turn-off the external electronic device(or some elements), adjusting the brightness (or resolution) of thedisplay, etc.) of the external electronic device communicating with theelectronic device or an application operating on the external electronicdevice. According to an embodiment of the present disclosure, theapplication 970 may include an application received from the externalelectronic device. At least a portion of the program module 910 may beimplemented (e.g., executed) in software, firmware, hardware (e.g., theprocessor 210), or a combination of at least two or more thereof and mayinclude a module, program, routine, command set, or process forperforming one or more functions.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise identifying a motion of an externalobject using a first image obtained by controlling the camera, obtainingfirst direction information based on the identified motion of theexternal object, driving at least one motor of the electronic device sothat the camera faces a direction determined according to the firstdirection information, and identifying a second electronic device from asecond image obtained by controlling the camera facing the determineddirection.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise transmitting data corresponding to thesecond image to at least one server through a communication interface ofthe electronic device, receiving data corresponding to the secondelectronic device determined based on the data corresponding to thesecond image from the at least one server, and identifying the secondelectronic device based on the received data corresponding to the secondelectronic device.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise receiving at least one voice input,transmitting data corresponding to the at least one voice input to atleast one server through a communication interface of the electronicdevice, receiving data corresponding to the second electronic devicedetermined based on the data corresponding to the at least one voiceinput from the at least one server, and identifying the secondelectronic device based on the received data corresponding to the secondelectronic device.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise, when the second electronic device isdetermined to be a controllable electronic device, obtaining datacorresponding to the second electronic device from a memory of theelectronic device, connecting with the second electronic device througha communication interface of the electronic device using the obtaineddata, and controlling the second electronic device.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise, when the data corresponding to thesecond electronic device is not in the memory, receiving the datacorresponding to the second electronic device from at least one serverthrough the communication interface, connecting with the secondelectronic device using the data received from the at least one server,and storing the obtained first direction information and the datacorresponding to the second electronic device received from the at leastone server in the memory.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise receiving at least one voice input,obtaining a command related to control of the second electronic devicefrom the at least one voice input, and transmitting a signalcorresponding to the obtained command to the second electronic device.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise identifying a state change in the secondelectronic device using a third image obtained by controlling the camerafacing the direction, and when the identified state change in the secondelectronic device does not correspond to the obtained command,outputting feedback corresponding to the at least one voice input.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise outputting a request related todesignation of a command corresponding to the second electronic device,storing the obtained first direction information and a commandcorresponding to the request in a memory of the electronic device, andupon receiving at least one voice input including the command stored inthe memory, transmitting a signal corresponding to the command to thesecond electronic device.

According to an embodiment of the present disclosure, a method forcontrolling, through an electronic device including a camera, anotherelectronic device may comprise, upon detecting a motion of the secondelectronic device, driving the at least one motor to trace the motion ofthe second electronic device, obtaining second direction informationabout the second electronic device based on a result of tracing themotion of the second electronic device, and storing the obtained seconddirection information in a memory of the electronic device.

According to an embodiment of the present disclosure, there may beprovided a computer readable recording medium storing a program forexecuting a method for controlling, through an electronic deviceincluding a camera, another electronic device, the method comprisingidentifying a motion of an external object using a first image obtainedby controlling the camera, obtaining first direction information basedon the identified motion of the external object, driving at least onemotor of the electronic device so that the camera faces a directiondetermined according to the first direction information, and identifyinga second electronic device from a second image obtained by controllingthe camera facing the direction.

As used herein, the term “module” includes a unit configured inhardware, software, or firmware and may interchangeably be used withother terms, e.g., “logic,” “logic block,” “part,” or “circuit.” Themodule may be implemented mechanically or electronically and mayinclude, e.g., an application-specific integrated circuit (ASIC) chip,field-programmable gate arrays (FPGAs), or programmable logic device,that has been known or to be developed in the future as performing someoperations.

According to an embodiment of the present disclosure, at least a part ofthe device (e.g., modules or their functions) or method (e.g.,operations) may be implemented as instructions stored in acomputer-readable storage medium (e.g., the memory 130), e.g., in theform of a program module. The instructions, when executed by a processor(e.g., the processor 120), may enable the processor to carry out acorresponding function. The computer-readable medium may include, e.g.,a hard disk, a floppy disc, a magnetic medium (e.g., magnetic tape), anoptical recording medium (e.g., CD-ROM, DVD, magnetic-optical medium(e.g., floptical disk), or an embedded memory. The instructions mayinclude a code created by a compiler or a code executable by aninterpreter. Modules or programming modules in accordance with variousembodiments of the present disclosure may include at least one or moreof the aforementioned components, omit some of them, or further includeother additional components. Operations performed by modules,programming modules or other components in accordance with variousembodiments of the present disclosure may be carried out sequentially,in parallel, repeatedly or heuristically, or at least some operationsmay be executed in a different order or omitted or other operations maybe added.

As is apparent from the foregoing description, according to anembodiment of the present disclosure, an electronic device may identifya user's motion and determine a particular object which the user desiresto control based on the user's motion.

According to an embodiment of the present disclosure, the electronicdevice may identify at least one object through vision-basedrecognition. The electronic device may move or turn to at least oneobject.

The embodiments disclosed herein are proposed for description andunderstanding of the disclosed technology and does not limit the scopeof the present disclosure. Accordingly, the scope of the presentdisclosure should be interpreted as including all changes or variousembodiments based on the technical spirit of the present disclosure.

The above-described embodiments of the present disclosure can beimplemented in hardware, firmware or via the execution of software orcomputer code that can be stored in a recording medium such as a CD ROM,a Digital Versatile Disc (DVD), a magnetic tape, a RAM, a floppy disk, ahard disk, or a magneto-optical disk or computer code downloaded over anetwork originally stored on a remote recording medium or anon-transitory machine readable medium and to be stored on a localrecording medium, so that the methods described herein can be renderedvia such software that is stored on the recording medium using a generalpurpose computer, or a special processor or in programmable or dedicatedhardware, such as an ASIC or FPGA. As would be understood in the art,the computer, the processor, microprocessor controller or theprogrammable hardware include memory components, e.g., RAM, ROM, Flash,etc. that may store or receive software or computer code that whenaccessed and executed by the computer, processor or hardware implementthe processing methods described herein.

What is claimed is:
 1. An electronic device, comprising: a camera; atleast one motor; a communication interface; a microphone; at least oneat least one processor; and a memory electrically connected with the atleast one processor, wherein the memory stores commands that, whenexecuted by the at least one processor, cause the at least one processorto: obtain voice input including information related to an externalobject through the microphone, recognize, based on the voice input, theexternal object, drive the at least one motor such that the camera facesa direction corresponding to a position of the external object, andidentify the external object, based on an image obtained through thecamera facing the direction.
 2. The electronic device of claim 1,wherein the voice input includes at least one of the position of theexternal object, a shape of the external object, or a model name of theexternal object.
 3. The electronic device of claim 1, wherein the memorystores commands that, when executed by the at least one processor, causethe at least one processor to: transmit first data corresponding to thevoice input to at least one server through the communication interface,receive second data corresponding to the external object from the atleast one server, the second data is determined based on the first data,and recognize the external object, based on the second datacorresponding to the external object.
 4. The electronic device of claim1, wherein the memory stores commands that, when executed by the atleast one processor, cause the at least one processor to: compare theinformation related to the external object with information stored inthe memory, and recognize, based on the comparison result, the externalobject.
 5. The electronic device of claim 1, wherein the memory storescommands that, when executed by the at least one processor, cause the atleast one processor to: transmit first data corresponding to the imageto at least one server through the communication interface, receivesecond data corresponding to the external object from the at least oneserver, wherein the second data is determined based on the first data,and identify the external object based on the second data correspondingto the external object.
 6. The electronic device of claim 1, wherein thememory stores commands that, when executed by the at least oneprocessor, cause the at least one processor to: when the external objectis determined to be a controllable electronic device, obtain datacorresponding to the external object from the memory, connect theelectronic device with the external object through the communicationinterface using the obtained data, and control the external object. 7.The electronic device of claim 6, wherein the memory stores commandsthat, when executed by the at least one processor, cause the at leastone processor to: when the data corresponding to the external object isnot in the memory, receive the data corresponding to the external objectfrom at least one server through the communication interface, andconnect the electronic device with the external object using the datareceived from the at least one server.
 8. The electronic device of claim6, wherein the memory stores commands that, when executed by the atleast one processor, cause the at least one processor to: obtain acommand related to control of the external object from the voice input,and transmit a signal corresponding to the obtained command to theexternal object.
 9. The electronic device of claim 8, wherein the memorystores commands that, when executed by the at least one processor, causethe at least one processor to: obtain an image of the external objectthrough the camera after transmitting the signal, and identify, based onthe image obtained before transmitting the signal and the image obtainedafter transmitting the signal, whether the external object performs afunction corresponding to the command.
 10. The electronic device ofclaim 1, wherein the memory stores commands that, when executed by theat least one processor, cause the at least one processor to: identifycoordinates of the external object corresponding to the position of theexternal object in a map of a space in which the electronic device islocated, and drive the at least one motor such that the camera faces adirection corresponding to the coordinates.
 11. A method forcontrolling, through an electronic device including a camera, anexternal object, the method comprising: obtaining voice input includinginformation related to the external object through a microphone of theelectronic device; recognizing, based on the voice input, the externalobject; driving at least one motor of the electronic device such thatthe camera faces a direction corresponding to a position of the externalobject; and identifying the external object, based on an image obtainedthrough the camera facing the direction.
 12. The method of claim 11,wherein the voice input includes at least one of the position of theexternal object, a shape of the external object, or a model name of theexternal object.
 13. The method of claim 11, wherein recognizing theexternal object comprises: transmitting first data corresponding to thevoice input to at least one server through a communication interface ofthe electronic device; receiving second data corresponding to theexternal object from the at least one server, the second data isdetermined based on the first data; and recognizing the external object,based on the second data corresponding to the external object.
 14. Themethod of claim 11, wherein recognizing the external object comprisescomparing the information related to the external object withinformation stored in the memory; and recognizing, based on thecomparison result, the external object
 15. The method of claim 11,wherein identifying the external object: transmitting first datacorresponding to the image to the at least one server through acommunication interface of the electronic device; receiving second datacorresponding to the external object from the at least one server,wherein the second data is determined based on the first data; andidentifying the external object based on the second data correspondingto the external object
 16. The method of claim 11, further comprising:when the external object is determined to be a controllable electronicdevice, obtaining data corresponding to the external object from amemory of the electronic device; connecting the electronic device withthe external object through communication interface of the electronicdevice using the obtained data; and controlling the external object. 17.The method of claim 16, further comprising: when the data correspondingto the external object is not in the memory, receiving the datacorresponding to the external object from at least one server through acommunication interface of the electronic device; and connecting theelectronic device with the external object using the data received fromthe at least one server.
 18. The method of claim 11, further comprising:obtaining a command related to control of the external object from thevoice input; and transmitting a signal corresponding to the obtainedcommand to the external object.
 19. The method of claim 11, furthercomprising: identifying coordinates of the external object correspondingto the position of the external object in a map of a space in which theelectronic device is located, wherein driving the at least one motorcomprises driving the at least one motor such that the camera faces adirection corresponding to the coordinates.
 20. A non-transitorycomputer readable recording medium storing a program for executing amethod for controlling, through an electronic device including a camera,an external object, the method comprising: obtaining voice inputincluding information related to the external object through amicrophone of the electronic device; recognizing, based on the voiceinput, the external object; driving at least one motor of the electronicdevice such that the camera faces a direction corresponding to aposition of the external object; and identifying the external object,based on an image obtained through the camera facing the direction.